<html>
<head><meta charset="utf-8"><title>Creating a safe interface to Thin Trait Objects · t-lang/wg-unsafe-code-guidelines · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/index.html">t-lang/wg-unsafe-code-guidelines</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects.html">Creating a safe interface to Thin Trait Objects</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="220504725"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating%20a%20safe%20interface%20to%20Thin%20Trait%20Objects/near/220504725" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Michael Bryan <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects.html#220504725">(Dec 20 2020 at 05:44)</a>:</h4>
<p>Earlier in the week I wrote up <a href="http://adventures.michaelfbryan.com/posts/ffi-safe-polymorphism-in-rust/">a blog post</a> about implementing "Thin Trait Objects" and someone mentioned that I might want to also ask for a code review here.  </p>
<p>I went through a couple iterations while writing up the article and <a href="https://github.com/Michael-F-Bryan/thin-trait-objects/">supporting code</a>, and also got feedback on the draft article on the community discord so I'm fairly confident there aren't any glaring logic errors.</p>
<p>The main question I have is around provenance and how it affects the way I've implemented "C-style inheritance" and then created a safe wrapper around the <code>unsafe</code> code and raw pointers... In particular, is it valid to upcast from a <code>*mut Child</code> to a <code>*mut Parent</code>, then later on downcast back to a <code>*mut Child</code>? If I upcast to a <code>*mut Parent</code> then pass that pointer across the FFI boundary, will it have "forgotten" that it used to actually point to a <code>*mut Child</code> allocation.</p>
<p>For a concrete example here's a fairly typical chain of events, calling a virtual method:</p>
<ol>
<li>Create a <code>Box&lt;Repr&lt;W&gt;&gt;</code> and use <code>Box::into_raw()</code> to convert it to a raw pointer (<a href="https://github.com/Michael-F-Bryan/thin-trait-objects/blob/b71b22c2c83fd07126c6274fa86f54a148afb580/src/file_handle.rs#L36-L41">src</a>)</li>
<li>Cast the <code>*mut Repr&lt;W&gt;</code> to a <code>*mut FileHandle</code> (<code>FileHandle</code> is the first field on <code>Repr&lt;W&gt;</code> and both are <code>#[repr(C)]</code>) (<a href="https://github.com/Michael-F-Bryan/thin-trait-objects/blob/b71b22c2c83fd07126c6274fa86f54a148afb580/src/file_handle.rs#L43-L45">src</a>)</li>
<li>That raw pointer is then somehow passed to the <code>OwnedFileHandle::from_raw()</code> constructor (<a href="https://github.com/Michael-F-Bryan/thin-trait-objects/blob/b71b22c2c83fd07126c6274fa86f54a148afb580/src/owned.rs#L47">src</a>)</li>
<li>We implement the <code>write()</code> method from <code>std::io::Write</code> on <code>OwnedFileHandle</code> by reading the <code>write</code> function from the <code>NonNull&lt;FileHandle&gt;</code> then calling it with our <code>FileHandle</code> pointer and the <code>&amp;[u8]</code> buffer (<a href="https://github.com/Michael-F-Bryan/thin-trait-objects/blob/b71b22c2c83fd07126c6274fa86f54a148afb580/src/owned.rs#L114-L118">src</a>)</li>
<li>The <code>write</code> function is actually a generic <code>unsafe fn write&lt;W&gt;(*mut FileHandle, &amp;[u8]) -&gt; std::io::Result&lt;usize&gt; where W: Write</code> which was instantiated for whatever <code>W</code> type was passed to <code>FileHandle::for_writer()</code> (<a href="https://github.com/Michael-F-Bryan/thin-trait-objects/blob/b71b22c2c83fd07126c6274fa86f54a148afb580/src/file_handle.rs#L70-L73">src</a>)</li>
<li>Inside <code>write</code> we cast the <code>*mut FileHandle</code> back to the <code>*mut Repr&lt;W&gt;</code> that was originally constructed, turn it into a mutable reference, then call the original writer's <code>write()</code> method</li>
</ol>
<p>I've tried to link to the code where I can, but here are the relevant type definitions: </p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="cp">#[repr(C)]</span><span class="w"></span>
<span class="k">pub</span><span class="w"> </span><span class="k">struct</span> <span class="nc">FileHandle</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">pub</span><span class="p">(</span><span class="k">crate</span><span class="p">)</span><span class="w"> </span><span class="n">layout</span>: <span class="nc">Layout</span><span class="p">,</span><span class="w"></span>
<span class="w">    </span><span class="k">pub</span><span class="p">(</span><span class="k">crate</span><span class="p">)</span><span class="w"> </span><span class="n">type_id</span>: <span class="nc">TypeId</span><span class="p">,</span><span class="w"></span>
<span class="w">    </span><span class="k">pub</span><span class="p">(</span><span class="k">crate</span><span class="p">)</span><span class="w"> </span><span class="n">destroy</span>: <span class="nc">unsafe</span><span class="w"> </span><span class="k">fn</span><span class="p">(</span><span class="o">*</span><span class="k">mut</span><span class="w"> </span><span class="n">FileHandle</span><span class="p">),</span><span class="w"></span>
<span class="w">    </span><span class="k">pub</span><span class="p">(</span><span class="k">crate</span><span class="p">)</span><span class="w"> </span><span class="n">write</span>: <span class="nc">unsafe</span><span class="w"> </span><span class="k">fn</span><span class="p">(</span><span class="o">*</span><span class="k">mut</span><span class="w"> </span><span class="n">FileHandle</span><span class="p">,</span><span class="w"> </span><span class="o">&amp;</span><span class="p">[</span><span class="kt">u8</span><span class="p">])</span><span class="w"> </span>-&gt; <span class="nb">Result</span><span class="o">&lt;</span><span class="kt">usize</span><span class="p">,</span><span class="w"> </span><span class="n">Error</span><span class="o">&gt;</span><span class="p">,</span><span class="w"></span>
<span class="w">    </span><span class="k">pub</span><span class="p">(</span><span class="k">crate</span><span class="p">)</span><span class="w"> </span><span class="n">flush</span>: <span class="nc">unsafe</span><span class="w"> </span><span class="k">fn</span><span class="p">(</span><span class="o">*</span><span class="k">mut</span><span class="w"> </span><span class="n">FileHandle</span><span class="p">)</span><span class="w"> </span>-&gt; <span class="nb">Result</span><span class="o">&lt;</span><span class="p">(),</span><span class="w"> </span><span class="n">Error</span><span class="o">&gt;</span><span class="p">,</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="cp">#[repr(C)]</span><span class="w"></span>
<span class="k">pub</span><span class="p">(</span><span class="k">crate</span><span class="p">)</span><span class="w"> </span><span class="k">struct</span> <span class="nc">Repr</span><span class="o">&lt;</span><span class="n">W</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="c1">// Safety: The FileHandle must be the first field so we can cast between</span>
<span class="w">    </span><span class="c1">// *mut Repr&lt;W&gt; and *mut FileHandle</span>
<span class="w">    </span><span class="k">pub</span><span class="p">(</span><span class="k">crate</span><span class="p">)</span><span class="w"> </span><span class="n">base</span>: <span class="nc">FileHandle</span><span class="p">,</span><span class="w"></span>
<span class="w">    </span><span class="k">pub</span><span class="p">(</span><span class="k">crate</span><span class="p">)</span><span class="w"> </span><span class="n">writer</span>: <span class="nc">W</span><span class="p">,</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="cp">#[repr(transparent)]</span><span class="w"></span>
<span class="k">pub</span><span class="w"> </span><span class="k">struct</span> <span class="nc">OwnedFileHandle</span><span class="p">(</span><span class="n">NonNull</span><span class="o">&lt;</span><span class="n">FileHandle</span><span class="o">&gt;</span><span class="p">);</span><span class="w"></span>
</code></pre></div>



<a name="220518255"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating%20a%20safe%20interface%20to%20Thin%20Trait%20Objects/near/220518255" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects.html#220518255">(Dec 20 2020 at 12:30)</a>:</h4>
<blockquote>
<p>In particular, is it valid to upcast from a *mut Child to a *mut Parent, then later on downcast back to a *mut Child? If I upcast to a *mut Parent then pass that pointer across the FFI boundary, will it have "forgotten" that it used to actually point to a *mut Child allocation.</p>
</blockquote>
<p>This is valid <em>if there is never a reference being created in this chain</em>. Raw pointers just maintain the provenance of whatever they were created from.</p>



<a name="220518272"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating%20a%20safe%20interface%20to%20Thin%20Trait%20Objects/near/220518272" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects.html#220518272">(Dec 20 2020 at 12:31)</a>:</h4>
<p>note that <code>&amp;ptr.field as *const _</code> creates a reference (and then casts it to a ptr), so such an interaction would be a problem</p>



<a name="220518283"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating%20a%20safe%20interface%20to%20Thin%20Trait%20Objects/near/220518283" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects.html#220518283">(Dec 20 2020 at 12:31)</a>:</h4>
<p>you can test this in Miri by passing <code>-Zmiri-track-raw-pointers</code>, which enables a "strict mode" where int-to-ptr casts don't really work but provenance is tracked throughout</p>



<a name="220523134"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating%20a%20safe%20interface%20to%20Thin%20Trait%20Objects/near/220523134" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Michael Bryan <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects.html#220523134">(Dec 20 2020 at 14:50)</a>:</h4>
<p><span class="user-mention silent" data-user-id="120791">RalfJ</span> <a href="#narrow/stream/136281-t-lang.2Fwg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects/near/220518255">said</a>:</p>
<blockquote>
<blockquote>
<p>In particular, is it valid to upcast from a *mut Child to a *mut Parent, then later on downcast back to a *mut Child? If I upcast to a *mut Parent then pass that pointer across the FFI boundary, will it have "forgotten" that it used to actually point to a *mut Child allocation.</p>
</blockquote>
<p>This is valid <em>if there is never a reference being created in this chain</em>. Raw pointers just maintain the provenance of whatever they were created from.</p>
</blockquote>
<p>Awesome! That was my feeling, so I went to great lengths to always use raw pointers for every operation inside my safe wrapper.</p>



<a name="220523149"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating%20a%20safe%20interface%20to%20Thin%20Trait%20Objects/near/220523149" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Michael Bryan <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects.html#220523149">(Dec 20 2020 at 14:50)</a>:</h4>
<p><span class="user-mention silent" data-user-id="120791">RalfJ</span> <a href="#narrow/stream/136281-t-lang.2Fwg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects/near/220518283">said</a>:</p>
<blockquote>
<p>you can test this in Miri by passing <code>-Zmiri-track-raw-pointers</code>, which enables a "strict mode" where int-to-ptr casts don't really work but provenance is tracked throughout</p>
</blockquote>
<p>I ended up needing to make a dummy <code>main.rs</code> because <code>-Zmiri-track-raw-pointers</code> didn't like the <code>std::sync::mpsc</code> code used by Rust's test runner (<a href="https://github.com/rust-lang/rust/issues/80234">rust-lang/rust#80234</a>) but once I got the code running it finished without a complaint, which gives me more confidence <span aria-label="smiling face" class="emoji emoji-263a" role="img" title="smiling face">:smiling_face:</span></p>



<a name="220523208"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating%20a%20safe%20interface%20to%20Thin%20Trait%20Objects/near/220523208" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Michael Bryan <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects.html#220523208">(Dec 20 2020 at 14:52)</a>:</h4>
<p><span class="user-mention silent" data-user-id="120791">RalfJ</span> <a href="#narrow/stream/136281-t-lang.2Fwg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects/near/220518272">said</a>:</p>
<blockquote>
<p>note that <code>&amp;ptr.field as *const _</code> creates a reference (and then casts it to a ptr), so such an interaction would be a problem</p>
</blockquote>
<p>What does that mean for code where you go from a pointer to an object to a pointer to one of its fields? In pretty much every <code>unsafe</code> Rust I've seen we need to create a temporary <code>&amp;mut</code> reference.</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="k">unsafe</span><span class="w"> </span><span class="k">fn</span> <span class="nf">frobnicate</span><span class="p">(</span><span class="n">foo</span>: <span class="o">*</span><span class="k">mut</span><span class="w"> </span><span class="n">Foo</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">  </span><span class="kd">let</span><span class="w"> </span><span class="k">mut</span><span class="w"> </span><span class="n">first</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="p">(</span><span class="o">*</span><span class="n">foo</span><span class="p">).</span><span class="n">bar</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="o">*</span><span class="k">mut</span><span class="w"> </span><span class="n">Bar</span><span class="p">;</span><span class="w"></span>

<span class="w">  </span><span class="o">..</span><span class="p">.</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>Even if it's immediately cast to a <code>*mut</code> pointer, the mere creation of a <code>&amp;mut</code> reference should mean we need to uphold reference invariants, doesn't it?</p>



<a name="220525634"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating%20a%20safe%20interface%20to%20Thin%20Trait%20Objects/near/220525634" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects.html#220525634">(Dec 20 2020 at 16:02)</a>:</h4>
<blockquote>
<p>Even if it's immediately cast to a *mut pointer, the mere creation of a &amp;mut reference should mean we need to uphold reference invariants, doesn't it?</p>
</blockquote>
<p>It means that you briefly assert reference invariants, and then weaken to a raw pointer <em>for the new type</em>. This causes <a href="https://github.com/rust-lang/unsafe-code-guidelines/issues/134">problems</a>, but it's the (experimental) rules we currently got. I hope at some point we'll have better rules. :)</p>



<a name="220525672"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating%20a%20safe%20interface%20to%20Thin%20Trait%20Objects/near/220525672" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects.html#220525672">(Dec 20 2020 at 16:03)</a>:</h4>
<blockquote>
<p>What does that mean for code where you go from a pointer to an object to a pointer to one of its fields?</p>
</blockquote>
<p>It means you should be using <code>ptr::raw_const!/raw_mut!</code>. Which unfortunately isn't stable yet... <a href="https://github.com/rust-lang/rust/issues/73394">FCP is done</a>, but rustdoc <a href="https://github.com/rust-lang/rust/issues/74355">has a bug</a> that makes it currently infeasible to stabilize the macro... :/<br>
(Out of all the barriers this macro had to overcome -- it's been a long time in the making -- this is the one that I least expected.^^ And the most frustrating thing is that fixing it is waay outside my area of expertise.)</p>



<a name="220534685"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating%20a%20safe%20interface%20to%20Thin%20Trait%20Objects/near/220534685" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Michael Bryan <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects.html#220534685">(Dec 20 2020 at 19:56)</a>:</h4>
<p><span class="user-mention silent" data-user-id="120791">RalfJ</span> <a href="#narrow/stream/136281-t-lang.2Fwg-unsafe-code-guidelines/topic/Creating.20a.20safe.20interface.20to.20Thin.20Trait.20Objects/near/220525672">said</a>:</p>
<blockquote>
<blockquote>
<p>What does that mean for code where you go from a pointer to an object to a pointer to one of its fields?</p>
</blockquote>
<p>It means you should be using <code>ptr::raw_const!/raw_mut!</code>. Which unfortunately isn't stable yet... <a href="https://github.com/rust-lang/rust/issues/73394">FCP is done</a>, but rustdoc <a href="https://github.com/rust-lang/rust/issues/74355">has a bug</a> that makes it currently infeasible to stabilize the macro... :/<br>
(Out of all the barriers this macro had to overcome -- it's been a long time in the making -- this is the one that I least expected.^^ And the most frustrating thing is that fixing it is waay outside my area of expertise.)</p>
</blockquote>
<p>That's awesome to hear!</p>
<p>I remember talking with dtolnay about the need for some sort of <code>&amp;raw foo.bar.baz</code> reference almost a year ago so it's nice to see progress has been made towards the idea. I also like that you are going with a macro so the feature can be proven in the real world instead of getting bogged down on syntax for months on end <span aria-label="smile" class="emoji emoji-1f642" role="img" title="smile">:smile:</span></p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>